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ABSTRACT 



This study sought to examine the effects of meta cognitive 
self-regulation on problem solving across three conditions: (1) an 

interactive, computer-based treatment condition; (2) a noninteractive 
computer-based alternative treatment condition; and (3) a control condition. 
Also investigated was which of five components of metacognitive self- 
regulation were important for scientific problem solving. It was hypothesized 
that overall metacognitive self-regulation and its various components would 
predict success at content understanding and problem solving, and that the 
treatment condition would be more effective in promoting learning outcomes 
than either the alternative treatment or control conditions. Participants 
included 626 students in grades 5 through 12 from schools across the United 
States. In all, 12 hierarchical linear models were produced. Results indicate 
that students in the treatment condition demonstrated significantly more 
content understanding and problem solving skill than students in the 
alternative and control classrooms. In the treatment condition, of the five 
components, only problem representation was a significant predictor for 
success at content understanding. In the alternative condition, students' 
problem representation had a significant inverse influence on content 
understanding. In terms of problem solving, knowledge of cognition and 
problem representation were found to be significant predictors. The 
implications of these results for instruction are discussed. (Contains 1 
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Abstract 

First, we sought to examine the effects of metacognilive self- regulation on problem solving across three conditions — an 
interactive, computer-based treatment condition, a non-interactive computer-based alternative treatment condition, and a control 
condition. Second, we sought to investigate which of five components of metacognitive self-regulation were important for 
scientific problem solving. We hypothesized that overall metacognitive self-regulation and its various components would predict 
success at content understanding and problem solving and that the treatment condition would be more effective in promoting 
learning outcomes than either the alternative treatment or control conditions. 

Overall, 12 hierarchical linear models were produced. Results indicated that students in the treatment condition 
demonstrated significantly more Content Understanding and Problem Solving skill than students in the alternative and control 
classrooms. In regards to the treatment condition, of the five IMSR components, only Problem Representation was a significant 
predictor for success at Content Understanding. In contrast, within the alternative condition, students' Problem Representation had 
a significant inverse influence on Content Understanding. In terms of Problem Solving, Knowledge of Cognition and Problem 
Representation were found to be significant predictors. 

These findings are especially noteworthy for science education and inquiry-based education. In particular, results indicate 
that metacognilive and self-regulatory constructs are important in teaching problem solving. Being able to identify and delineate 
these constructs further should allow our educational research and teacher professional development teams to begin providing 
teachers with a set of tools and training resources to help them target student self- regulation in their classrooms. 



Introduction 



Self-Regulated Learning. Metacognition, and Problem Solving 

Many researchers would agree that an important goal of education is the development of intellectual independence — the 
ability to think critically and solve the every-day problems of life. Studies on the development of self- regulated learning offer 
important insights into the complex interrelationships between cognitive, metacognitive, and affective aspects of intellectual 
independence. Research conducted over the last 15 years on self-regulated learning has primarily focused on three core 
components: metacognitive awareness, strategy use, and motivational control (Bruning, Schraw & Ronning, 1995). In the present 
study we focused on the influence of metacognitive awareness for effective use of problem solving strategies. 

Metacognition has been referred to as knowledge and regulation of one’s own cognitive system (Brown, 1978; Palincsar 
& Brown, 1987). Metacognition enables students to coordinate the use of current knowledge and a repertoire of reflective 
strategies to accomplish a single goal. Metacognitive awareness, therefore, serves a regulatory function and is essential to 
effective learning because it enables students to regulate numerous cognitive skills. 

Studies of metacognition in academic settings has traditionally focused on two major components: knowledge of 
cognition — how much learners understand about their own memory organization and the way they learn, — and regulation of 
IQ cognition — how well learners regulate their own memory and learning (Brown 1980; 1987). In an instrument development study, 

O Howard, McGee, Shia, and Hong (2000) confirmed the existence of a knowledge of cognition factor and two regulation of 

cognition factors which they titled sub task monitoring and evaluation . They also found two additional self- regulatory constructs 
pertinent to problem solving, problem representation and objectivity . 

q The current work, therefore, examines five components of metacognitive self-regulation (Howard, McGee, Shia, & Hong, 

g 2000): 

|_ • Knowledge of Cognition : understanding the extent and utilization of one’s unique cognitive abilities and the ways one 

learns best. 

• Suhtask Monitoring (regulation of cognition) : breaking the problem down into subtasks and monitoring the completion of 
each subtask. 



1 1 Contact: Bruce Howard or Steven McGee, 316 Washington Avenue. Wheeling, WV 26003, (304) 243-2388. howard@cet.edu, megee@eet.edu 
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• Evaluation (regulation of cognition) : double-checking throughout the entire problem-solving process to evaluate if it is 

being done correctly. 

• Problem Representation : understanding the problem fully before proceeding. 

• Objectivity : standing outside oneself and thinking about one’s learning as it proceeds. 

Metacognition and Problem Solving 

In 1990 H. Lee Swanson presented a pivotal work linking metacognition to successful problem solving. Swanson set out 
to demonstrate the independence of metacognition and general aptitude on various problem-solving measures. He measured 
aptitude with standardized, cognitive ability and achievement tests and metacognitive ability using tape-recorded responses to a 
melacognitive questionnaire. His findings indicated that metacognition was more important for problem-solving success than 
aptitude. In situations where students had low aptitudes but high metacognitive levels, students performed as well as students of 
high aptitude. 

Research in which students used a CD-ROM titled Astronomy Village®: Investigating the Universe'” provided evidence 
that students may not necessarily need nor use high levels of metacognilion to solve every type of problem (Hong, 1 998). This 
research indicated that metacognitive awareness was a significant predictor of success for ill-structured problem solving, but was 
not significant for solving well-structured problems. In addition, Howard, McGee, Hong and Shia (2000) found that three of the 
five factors (Knowledge of Cognition, Problem Representation, & Objectivity) were significant predictors of Content 
Understanding. In addition, four of five factors (Knowledge of Cognition, Evaluation, Problem Representation, & Objectivity) 
were significant predictors of Problem Solving. Results also showed that those with High Metacognitive Self-Regulation 
compensated for Low Aptitude on both Content Understanding and Problem Solving measures. 

Some could argue that metacognilion is innate and, therefore, largely unchangeable through instructional intervention. 
Research in science education, however, indicates that a variety of regulatory behaviors may be learned, and that such behaviors 
are beneficial for learning. For example, research has shown that certain behaviors lead to success in science education, such as 
identifying goals (Linn 1995), self-assessing (White & Frederikscn, 1995), planning (King, 1988; Scardamalia & Bereiter, 1991), 
self-explaining (Chi, Bassok, Lewis, Reimann, & Glaser 1989), self-questioning (King 1994), reflecting (Davis, 1998; Audet, 
Hickman & Dobrynina, 1996), and making concepts personally relevant (Linn, 1995). 

Metacognitive training has been shown to be particularly effective for the acquisition of reading (Jacobs & Paris, 1987; 
Palincsar & Brown, 1984) and problem solving strategies (Delclos & Harrington, 1991) regardless of aptitude or achievement 
level. However, further evidence that metacognilion affects variables that influence learning is scant. For instance Pintrich, Smith, 
Garcia, and McKeachie (1991) indicated that the use of metacognitive and cognitive strategies was not highly correlated with 
academic achievement. Pressley and Ghatala (1988) also found metacognilion (in this case monitoring accuracy) to be unrelated 
to verbal ability. 

Research Questions 

We examined the effects of metacognitive self-regulation on problem solving across three conditions in 36 classrooms. In 
the treatment condition, students learned science using interactive, computer-based software. We hypothesized that metacognitive 
self-regulation would predict success at problem solving. In the alternative treatment condition, students used non-interactive 
computer-based materials, and completed associated worksheets. The control condition students completed pre- and posttesls but 
did not complete any relevant instruction. 

In this study, we also sought to investigate which of the five components of metacognitive self- regulation were important 
for scientific problem solving. The results would be important for creating a descriptive profile of the components of 
metacognitive self-regulation that are most necessary for problem solving. We hypothesized that overall metacognitive self- 
regulation and its various components would predict success at content understanding and problem solving and that the treatment 
condition would be more effective in promoting learning outcomes than either the alternative treatment or control conditions. 



Method 

Participants 

Participants included 626 students, grades 5-12, from schools across the United States. They represented a cross-section 
of socioeconomic backgrounds and urban/suburban/rural categorizations. The ethnic breakdown of treatment and alternative 
treatment conditions included 65.5% Caucasian, 24. 1 % Asian American, 3.6% African American, 4.1% Hispanic or Latino, and 
2.7% Other. By gender, the breakdown was 50% female and 50% male. 



Procedure/ Materials 

In the treatment condition, students used the Astronomy Village® Investigating the Solar System™ software. In the 
alternative treatment condition, students had access to the same content on the computer, but without the benefit of the Village 
interface and image analysis activities. Each group covered the material for an average of 20 instructional days. 

Students were given pretest/posttest instruments that measured learning: One instrument measured Content 
Understanding , and the other measured Problem Solving (see McGee & Howard, 1999 for description of Astronomy Village). 
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Pretest scores were subtracted from the posltest scores to yield one score that represents the amount of learning gained from each 
student’s instructional experience. 

At pretest time students also took the Inventory of Melacognitive Self-Regulation (IMSR) which measures five factors 
related to awareness of learning processes and control of learning strategies: (1) Knowledge Of Cognition, (2) Subtask Monitoring, 
(3) Evaluation (4) Problem Representation, and (5) Objectivity (Howard, McGee, Sliia, & Hong, 2000). The IMSR includes 32 
items that use a five-point Likert scale. For each of the 32 items, students are instructed to circle the answer that best described 
"the way they are" when solving problems in math or science class (l=never, 2=seldom/rarely, 3=sometimes, 4=often/frequently, 
5=always). The validation of the IMSR and a more detailed explanation of the five components is discussed elsewhere (Howard, 
McGee, Sliia, & Hong, 2000). 

Hierarchical Linear Modeling 

We chose to use a data analysis technique known as hierarchical linear modeling (HLM), which has several advantages 
over ordinary least squares (OLS) regression, in that it allows analyses to be conducted simultaneously at multiple levels of data. 
Variables such as teacher effects, class period, and student ability or altitudinal levels can influence individual performance. When 
using OLS regression model, such variables modify the classroom or teacher-level outcomes, leaving unchanged the distribution ot 
effects among individuals. In OLS analyses, only the intercept of a particular variable changes when predicting scores on a 
dependent variable. 

To combat this problem, HLM uses a random-intercept model where the classroom or teacher-level effects modify both 
the classroom or teacher-level outcome and how these effects are distributed among individuals. HLM also reduces the chances of 
making a type I error. Measuring the effect of a variable at the student level ignores the tact that these students are nested within 
their classroom, resulting in an estimated standard error that is exagerratedly, thus inflating Type 1 errors (Aitkin, Anderson, & 
Hinde, 1981). HLM also uniquely shows how variables at one level influence relations occurring at another level (Bryk & 
Raudenbush, 1992). 



Results 

Overall, 12 hierarchical linear models were produced. Student-level variables included Total IMSR and the five 
components of the IMSR. Classroom-level variables were coded as dummy variables to compare: 1) treatment vs. control, 2) 
alternative treatment vs. control, and 3) treatment vs. alternative treatment. Classroom-level variables were included in each model 
with each student-level variable in separate analyses for both dependent variables. Results for all analysis are included in Table I . 

Overall, results indicated that students in the treatment condition demonstrated significantly more Content Understanding 
and Problem Solving skill than students in the alternative and control classrooms (p=<.0()l). Experimental condition did not 
influence the effect of Total IMSR on Content Understanding or Problem Solving. 

In regards to the treatment condition, of the five IMSR components, only Problem Representation was a significant 
predictor for success at Content Understanding, B^l .446, p=.047. In contrast, within the alternative condition, students' Problem 
Representation had a significant inverse influence on Content Understanding, B=-.885, p=.042. 

In terms of Problem Solving, Knowledge of Cognition and Problem Representation were found to be significant 
predictors, B=4.363, p=.()22 and B=3.847, p=.002 respectively. Though Monitoring abilities yielded a p value of .064, no other 
level-one variable was found to be significant. 



Implications 

These findings are especially noteworthy for science education and inquiry-based education. We hypothesized that 
overall metacognitive self-regulation und its various components would predict success at content understanding and problem 
solving and that the treatment condition would be more effective in promoting learning outcomes than either the alternative 
treatment or control conditions. 

In particular, our results show that while the use of the software was a significant predictor in all analyses, in some 
instances metacognitive self-regulatory abilities yielded a higher intercept when predicting scores on the dependent variables. 
Knowledge of Cognition and Problem Representation lent more contribution to the intercept of gain in Problem Solving skills than 
did the effect of the software. It could be that such metacognitive self- regulatory skills are so important for individual students that 
even classroom-level variables such as the type of instruction received did little to take away from this effect. 

In retrospect, we realize that two 11011 -significant variables. Subtask Monitoring and Evaluation, may not have been 
important for either Content Understanding or Problem Solving because the software accomplishes such tasks for the learner 
(therefore clouding the overall effect). That is, the program breaks down problems into manageable chunks and helps students 
monitor and evaluate completion of those chunks. This finding is in line with our prior research which demonstrated that well- 
organized knowledge helps students apply their content understanding in solving novel science problems (Hong, McGee & 
Howard, 1999). 
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Knowledge of Cognition was not a predictor for Content Understanding, which is in line with prior research indicating 
that nielacognition was a significant predictor for ill-structured problem solving, but not for well-structured problem solving 
(Hong, 1998). Problem Representation characterizes self-regulatory processes and, therefore, may be an important factor in 
predicting both Content Understanding and Problem Solving. Contrary to past research, Objectivity had no effect on the Content 
Understanding or Problem Solving. We suggest further investigation of the construct to verify its role in the learning process. 

From this study it can be concluded that metacognitive and self-regulatory constructs are important in teaching problem 
solving. Being able to identify and delineate these constructs further should allow our educational research and teacher 
professional development teams to begin providing teachers with a set of tools and training resources to help them target student 
self-regulation in their classrooms. 

Further, our analyses indicate that the constructs measured by the IMSR are independent, and therefore a student may 
show preferences or ‘‘styles” of metacognitive strengths and weaknesses. If these “styles” can be further understood and 
delineated, it might be possible to train students to habitually use particular regulatory behaviors. 
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Dependent Variable: Content Understanding 
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Objectivity 
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Problem Representation 
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Error 
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32 
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0.000 
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1.234447 


0.500079 


2.469 




32 


0.019 
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32 
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Knowledge of Cognition 



Fixed Effect 
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Standard 
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Error 
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MEANKC, G03 




1.460261 


1.098323 


1.330 




32 


0.193 


For KC slope, 


BL 
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32 
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0.315763 
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32 
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32 
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1.200096 
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32 
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Evaluation 








Standard 




Approx. 




Fixed Effect 




Coefficient 


Error 


T-ratio 


d.f. 




P-value 
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32 
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ALT, G02 




1.455396 


0.522918 
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32 
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32 
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32 
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32 
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32 
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